{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial: The Basic Tools of Private Deep Learning\n",
    "\n",
    "Welcome to PySyft's introductory tutorial for privacy preserving, decentralized deep learning. This series of notebooks is a step-by-step guide for you to get to know the new tools and techniques required for doing deep learning on secret/private data/models without centralizing them under one authority.\n",
    "\n",
    "**Scope:** Note that we'll not just be talking about how to decentralized / encrypt data, but we'll be addressing how PySyft can be used to help decentralize the entire ecosystem around data, even including the Databases where data is stored and queried, and the neural models which are used to extract information from data. As new extensions to PySyft are created, these notebooks will be extended with new tutorials to explain the new functionality.\n",
    "\n",
    "Authors:\n",
    "- Andrew Trask - Twitter: [@iamtrask](https://twitter.com/iamtrask)\n",
    "\n",
    "## Outline:\n",
    "\n",
    "- Part 1: The Basic Tools of Private Deep Learning\n",
    "\n",
    "\n",
    "## Why Take This Tutorial?\n",
    "\n",
    "**1) A Competitive Career Advantage** - For the past 20 years, the digital revolution has made data more and more accessible in ever larger quantities as analog processes have become digitized. However, with new regulation such as [GDPR](https://eugdpr.org/), enterprises are under pressure to have less freedom with how they use - and more importantly how they analyze - personal information. **Bottom Line:** Data Scientists aren't going to have access to as much data with \"old school\" tools, but by learning the tools of Private Deep Learning, YOU can be ahead of this curve and have a competitive advantage in your career. \n",
    "\n",
    "**2) Entrepreneurial Opportunities** - There are a whole host of problems in society that Deep Learning can solve, but many of the most important haven't been explored because it would require access to incredibly sensitive information about people (consider using Deep Learning to help people with mental or relationship issues!). Thus, learning Private Deep Learning unlocks a whole host of new startup opportunities for you which were not previously available to others without these toolsets.\n",
    "\n",
    "**3) Social Good** - Deep Learning can be used to solve a wide variety of problems in the real world, but Deep Learning on *personal information* is Deep Learning about people, *for people*. Learning how to do Deep Learning on data you don't own represents more than a career or entrepreneurial opportunity, it is the opportunity to help solve some of the most personal and important problems in people's lives - and to do it at scale.\n",
    "\n",
    "## How do I get extra credit?\n",
    "\n",
    "- Star PySyft on GitHub! - [https://github.com/OpenMined/PySyft](https://github.com/OpenMined/PySyft)\n",
    "- Make a Youtube video teaching this notebook!\n",
    "\n",
    "\n",
    "... ok ... let's do this!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part -1: Prerequisites\n",
    "\n",
    "- Know PyTorch - if not then take the http://fast.ai course and come back\n",
    "- Read the PySyft Framework Paper https://arxiv.org/pdf/1811.04017.pdf! This will give you a thorough background on how PySyft is constructed which will help things make more sense."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 0: Setup\n",
    "\n",
    "To begin, you'll need to make sure you have the right things installed. To do so, head on over to PySyft's readme and follow the setup instructions. TLDR for most folks is.\n",
    "\n",
    "- Install Python 3.6 or higher\n",
    "- Install PyTorch 1.4\n",
    "- pip install syft[udacity]\n",
    "\n",
    "If any part of this doesn't work for you - first check [INSTALLATION.md](https://github.com/OpenMined/PySyft/blob/master/INSTALLATION.md) for help, and then open a GitHub Issue or ping the #beginner channel in our Slack! [slack.openmined.org](http://slack.openmined.org/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell to see if things work\n",
    "import sys\n",
    "\n",
    "import torch\n",
    "from torch.nn import Parameter\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "import syft as sy\n",
    "hook = sy.TorchHook(torch)\n",
    "\n",
    "torch.tensor([1,2,3,4,5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If this cell executed, then you're off to the races! Let's do this!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 1: The Basic Tools of Private, Decentralized Data Science\n",
    "\n",
    "So - the first question you may be wondering is - How in the world do we train a model on data we don't have access to? \n",
    "\n",
    "Well, the answer is surprisingly simple. If you're used to working in PyTorch, then you're used to working with torch.Tensor objects like these!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor([1,2,3,4,5])\n",
    "y = x + x\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Obviously, using these super fancy (and powerful!) tensors is important, but also requires you to have the data on your local machine. This is where our journey begins. \n",
    "\n",
    "# Section 1.1 - Sending Tensors to Bob's Machine\n",
    "\n",
    "Whereas normally we would perform data science / deep learning on the machine which holds the data, now we want to perform this kind of computation on some **other** machine. More specifically, we can no longer simply assume that the data is on our local machine.\n",
    "\n",
    "Thus, instead of using Torch tensors, we're now going to work with **pointers** to tensors. Let me show you what I mean. First, let's create a \"pretend\" machine owned by a \"pretend\" person - we'll call him Bob."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bob = sy.VirtualWorker(hook, id=\"bob\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's say Bob's machine is on another planet - perhaps on Mars! But, at the moment the machine is empty. Let's create some data so that we can send it to Bob and learn about pointers!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor([1,2,3,4,5])\n",
    "y = torch.tensor([1,1,1,1,1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now - let's send our tensors to Bob!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_ptr = x.send(bob)\n",
    "y_ptr = y.send(bob)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_ptr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BOOM! Now Bob has two tensors! Don't believe me? Have a look for yourself!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bob._objects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = x_ptr + x_ptr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bob._objects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now notice something. When we called `x.send(bob)` it returned a new object that we called `x_ptr`. This is our first *pointer* to a tensor. Pointers to tensors do NOT actually hold data themselves. Instead, they simply contain metadata about a tensor (with data) stored on another machine. The purpose of these tensors is to give us an intuitive API to tell the other machine to compute functions using this tensor. Let's take a look at the metadata that pointers contain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_ptr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check out that metadata!\n",
    "\n",
    "There are two main attributes specific to pointers:\n",
    "\n",
    "- `x_ptr.location : bob`, the location, a reference to the location that the pointer is pointing to\n",
    "- `x_ptr.id_at_location : <random integer>`, the id where the tensor is stored at location\n",
    "\n",
    "They are printed in the format `<id_at_location>@<location>`\n",
    "\n",
    "There are also other more generic attributes:\n",
    "- `x_ptr.id : <random integer>`, the id of our pointer tensor, it was allocated randomly\n",
    "- `x_ptr.owner : \"me\"`, the worker which owns the pointer tensor, here it's the local worker, named \"me\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_ptr.location"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bob == x_ptr.location"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_ptr.id_at_location"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_ptr.owner"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may wonder why the local worker which owns the pointer is also a VirtualWorker, although we didn't create it.\n",
    "Fun fact, just like we had a VirtualWorker object for Bob, we (by default) always have one for us as well. This worker is automatically created when we called `hook = sy.TorchHook()` and so you don't usually have to create it yourself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "me = sy.local_worker\n",
    "me"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "me == x_ptr.owner"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And finally, just like we can call .send() on a tensor, we can call .get() on a pointer to a tensor to get it back!!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_ptr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_ptr.get()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_ptr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_ptr.get()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z.get()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bob._objects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And as you can see... Bob no longer has the tensors anymore!!! They've moved back to our machine!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Section 1.2 - Using Tensor Pointers\n",
    "\n",
    "So, sending and receiving tensors from Bob is great, but this is hardly Deep Learning! We want to be able to perform tensor _operations_ on remote tensors. Fortunately, tensor pointers make this quite easy! You can just use pointers like you would normal tensors!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor([1,2,3,4,5]).send(bob)\n",
    "y = torch.tensor([1,1,1,1,1]).send(bob)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = x + y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And voilà! \n",
    "\n",
    "Behind the scenes, something very powerful happened. Instead of x and y computing an addition locally, a command was serialized and sent to Bob, who performed the computation, created a tensor z, and then returned the pointer to z back to us!\n",
    "\n",
    "If we call .get() on the pointer, we will then receive the result back to our machine!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z.get()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Torch Functions\n",
    "\n",
    "This API has been extended to all of Torch's operations!!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = torch.add(x,y)\n",
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z.get()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variables (including backpropagation!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor([1,2,3,4,5.], requires_grad=True).send(bob)\n",
    "y = torch.tensor([1,1,1,1,1.], requires_grad=True).send(bob)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = (x + y).sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z.backward()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = x.get()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So as you can see, the API is really quite flexible and capable of performing nearly any operation you would normally perform in Torch on *remote data*. This lays the groundwork for our more advanced privacy preserving protocols such as Federated Learning, Secure Multi-Party Computation, and Differential Privacy !"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Congratulations!!! - Time to Join the Community!\n",
    "\n",
    "Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the movement toward privacy preserving, decentralized ownership of AI and the AI supply chain (data), you can do so in the following ways!\n",
    "\n",
    "### Star PySyft on GitHub\n",
    "\n",
    "The easiest way to help our community is just by starring the GitHub repos! This helps raise awareness of the cool tools we're building.\n",
    "\n",
    "- [Star PySyft](https://github.com/OpenMined/PySyft)\n",
    "\n",
    "### Join our Slack!\n",
    "\n",
    "The best way to keep up to date on the latest advancements is to join our community! You can do so by filling out the form at [http://slack.openmined.org](http://slack.openmined.org)\n",
    "\n",
    "### Join a Code Project!\n",
    "\n",
    "The best way to contribute to our community is to become a code contributor! At any time you can go to PySyft GitHub Issues page and filter for \"Projects\". This will show you all the top level Tickets giving an overview of what projects you can join! If you don't want to join a project, but you would like to do a bit of coding, you can also look for more \"one off\" mini-projects by searching for GitHub issues marked \"good first issue\".\n",
    "\n",
    "- [PySyft Projects](https://github.com/OpenMined/PySyft/issues?q=is%3Aopen+is%3Aissue+label%3AProject)\n",
    "- [Good First Issue Tickets](https://github.com/OpenMined/PySyft/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n",
    "\n",
    "### Donate\n",
    "\n",
    "If you don't have time to contribute to our codebase, but would still like to lend support, you can also become a Backer on our Open Collective. All donations go toward our web hosting and other community expenses such as hackathons and meetups!\n",
    "\n",
    "[OpenMined's Open Collective Page](https://opencollective.com/openmined)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
